Complex Event Detection in Multi-Modal Sensor Network

The Global War on Terror (GWOT) presents unique challenges in its intelligence requirements and the need exists to monitor at-risk individuals, groups, and installations. To detect these threats and create actionable intelligence to support expeditionary war fighting, there is a requirement for a network of sensors which can provide the required situational awareness. To that end, the goal of this paper is to detail the algorithms, tool-sets and interfaces to detect complex events from multi-modal sensor network and improve operator’s wide area situational awareness of live feeds from multiple aerial and ground sensors in the battlefield. The key innovations that will be discussed include: 1) Automated and robust detection of events and anomalies in video that may span multiple sensors; 2) Exploitation of scene context for improving event detection; 3) An interactive activity specification framework for effective threat specification and automatic recognition, tuned towards the requirements of the operator in the battlefield; 4) Automatic context-enabled wide area complex event detection by using information from multiple heterogeneous sensors; 5) Enhanced geo-browser user interfaces for wide area situational awareness of the operator. In addition, we provide information for system integration and discuss metrics for complex event detection evaluation, where we detail the thorough performance analysis that was carried out to ensure scalability and robustness of the system.